Cameras are designed to capture detailed spatial information from static scenes. However, problems arise if a camera or objects in a scene move during an exposure period. These problems are due to the nature of a sensor which integrates light received over a period of time to obtain an estimate of light intensity at each sensor element of the camera. The measured light intensity is used to generate an image of the scene. The period of time during which the sensor is exposed to incoming light is referred to as an exposure period or exposure time. Motion causes portions of the scene image or the entire scene image to move across the sensor, smearing the collected light across multiple sensor pixels. This smearing is commonly described by the term “motion blur”.
Whilst motion blur in a captured image can be used for artistic effect, it is generally viewed as a serious defect and resulting images are often discarded. To avoid blur, camera manufactures may incorporate hardware image stabilisation devices in a lens or at a sensor of a camera. However, hardware image stabilisation devices are only effective to counteract camera shake, and object motion in the scene still results in blur. Using a shorter exposure time reduces amount of blur but corresponding reduction in the amount of light captured can result in decreased image quality due to consequent increase in noise.
Post capture methods have been devised to reverse the blur-related smearing and create a de-blurred image. Such methods assume that the same blurred image can be obtained in two ways; firstly, through the use of a camera, and secondly by applying a blurring filter to an ideal snapshot image. The blurring is commonly modelled to be a linear, position-invariant process. Such a model allows the blurred image to be represented as a convolution with a blur kernel and some additive noise introduced by the capture process. Using the symbol * to express convolution and psf for a blurring filter kernel, also termed a point spread function (PSF), the blurred image, imageblurred, may be written in accordance with Equation (1) as follows:—imageblurred=psf*imagesnapshot+noise  (1)
In general terms, the point spread function describes how the light intensity of a point source at each location in the image is spread across neighbouring sensor pixels. A desired snapshot of the scene image may be obtained by inverting the action of the blurring operation. Direct inversion results in amplification of the noise, the severity of which depends on the nature of the point spread function and regularisation techniques are commonly employed to reduce this amplification. Typically, an instant of time chosen for the snapshot is the start, end or middle of the exposure period.
Two situations may occur in practical usage of a camera. Firstly, if the camera is not physically stabilised, camera shake results in blurring of a captured scene. In this case, an ideal image is an image of the scene as if the camera had not undergone any shaking. The second situation is where there is no camera shake but there are one or more objects moving in the scene. In this second situation, the ideal image is generally an image of the static scene with moving objects frozen in space, as if the objects were not moving.
Methods for removing blur in an image require an estimate of the point spread function. It is important that the estimate of the point spread function is accurate as errors can lead to serious artefacts, which can often be more objectionable to a viewer than original motion blur. Artefacts are easily introduced as the point spread function may be many pixels wide, and de-blurring will result in light from more distant pixels being added to local pixel values, leading to ghosting, if inaccuracies occur.
In a camera, it is usual for a sensor to be in a fixed position relative to a body of the camera. An image of a scene is captured by focusing light received by the camera from the scene onto the sensor for a finite exposure period. For a scene containing moving objects, the image captured by a fixed camera sensor will have regions of differing amounts of blur. The point spread function for the image varies for each moving object and depends on the object speed, direction and depth within the scene. This means that de-blurring methods need to segment the image of the scene and estimate the point spread function for each moving object separately. Segmenting the scene by identifying moving objects has been found to be a difficult problem. One issue is that the combination of a fixed exposure period and object motion with linear velocity results in a blur which overlaps a position of the object which is to be recovered in the image. Additionally, spatial frequency response of the corresponding blur point spread function is a “sinc” function which contains zeros. These zeros represent a complete loss of information concerning the image at the corresponding frequencies.
An alternative method of image capture has been proposed which attempts to avoid the need to segment the image according to objects of differing speeds. This alternative method has been termed “motion invariant imaging” and is applicable to scenes where objects have the same motion orientation. That is, objects that are moving on paths substantially parallel to each other in the image plane of a captured image. The method deliberately blurs captured sensor data by translating the image across the sensor of a camera used to capture the image during exposure time. In this instance, a single blur point spread function is used for de-blurring the captured image. A standard camera design which has been modified to achieve such a deliberate blur is referred to as a “motion invariant” camera. Ideally, in the motion invariant method, the point spread function is spatially invariant and is of a known form, containing only one parameter, the constant of acceleration, which is also known. The position of moving objects in the image does not correspond to one instant in time, but varies depending on object speed.